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From Sigmoid Power Control Algorithm to 

Hopfield-like Neural Networks: 
"SIR"-Balancing Sigmoid-Based Networks- 
Part II: Discrete Time 

Zekeriya Uykan 



Abstract 

In the first part in [12], we present and analyse a Sigmoid-based "Signal-to-lnterference Ratio, 
(SIR)" balancing dynamic network, called Sgm"SIR"NN, which exhibits similar properties as traditional 
Hopfield NN does, in continuous time. In this second part, we present the corresponding network in 
discrete time: We show that in the proposed discrete-time network, called D-Sgm"SIR"NN, the defined 
error vector approaches to zero in a finite step in both synchronous and asynchronous work modes. Our 
investigations show that i) Establishing an analogy to the distributed (sigmoid) power control algorithm 
in [10] and [11] if the defined fictitious "SIR" is equal to 1 at the converged eqiulibrium point, then it is 
one of the prototype vectors, ii) The D-Sgm"SlR"NN exhibits similar features as discrete-time Hopfield 
NN does, iii) Establishing an analogy to the traditional 1-bit fixed-step power control algorithm, the 
corresponding "1-bit" network, called Sign"SlR"NN network, is also presented. 

Index Terms 

Discrete-time Hopfield Network, distributed sigmoid power control algorithm. 

I. Introduction 

This paper is a continuation of the study in [12] where a continuous-time "Signal-to-Interference 
Ratio, (SIR)"-balancing neural network is presented which includes Hofield Network and sigmoid- 
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based power control algorithm algorithm of [10] and [11] as special cases, both of whose scopes 
of interest, motivations and settings are completely different. In this paper, we examine the 
discrete-time counterpart of [12], and propose two discrete-time sigmoid-basis SIR-balancing 
networks which exhibit similar features which generaly are attributed to recurrent neural networks 
like discrete-time Hopfield Networks. 

Hopfield Neural Networks has been an important focus of research area since early 1980s 
whose applications vary from combinatorial optimization (e.g. [2], [3] among many others) 
including traveling salesman problem (e.g. [4], [16] among others) to image restoration (e.g. 
[5]), from various control engineering optimization problems including in robotics (e.g. [8] 
among others) to associative memory systems (e.g. [7] among others), etc. For a tutorial and 
further references about Hopfield NN, see e.g. [13] and [9]. 

In the first part in [12], we present a Sigmoid-based "Signal-to-Interference Ratio (SIR)" 
balancing dynamic network, called Sgm"SIR"NN, which exhibits similar properties as traditional 
Hopfield NN does, is presented and analysed in continuous time. In this second part, we present 
the corresponding networks in discrete time. Our investigations show that i) Establishing an 
analogy to the distributed (sigmoid) power control algorithm in [10] and [11], if the defined 
fictitious "SIR" is equal to 1 at the converged eqiulibrium point, then it is one of the prototype 
vectors, ii) The D-Sgm"SIR"NN exhibits similar features as discrete-time Hopfield NN does, 
iii) Establishing an analogy to the traditional 1-bit fixed-step power control algorithm, the 
corresponding "1-bit" network, called Sign"SIR"NN network, is also presented. 

The paper is organized as follows: The proposed D-Sgm"SIR"NN and its 1-bit version network 
is presented and their stability features are analysed in section II. Simulation results are presented 
in Section EI followed by Concluding Remarks in Section IV. 

II. "SIR" -Balancing Sigmoid-Based Networks in Discrete Time 

We start with the standard definition of Signal-to-Interferende+Noise-Ratio (SIR) in a cellular 
radio system, in which N mobiles share the same channel (e.g. [17], [18]). 

9iiPi ■ 1 AT /1\ 

7i = — , t^l,...,N (1) 
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where pi is the transmission power of mobile i, gij is the link gain from mobile j to base i 
involving path loss, shadowing, multi-path fading (as well as the spreading/processing gain in 
case of CDMA transmission [6], etc), and Vi is the receiver noise at base station i. 

Because, in power control, the positive transmit power can not be arbitarily small and large 
in practice, we write the eq.(l) with the minimum and maximum power constraints as follows: 

Pi}} ■ 1 AT /ON 

where Pmin and Pmax is the minimum and maximum transmit powers. The SIR model in (2) can 
be further written in a more generalized equation as follows using neural networks termonilogy 

^. = guViPi) i=l N (3) 

^i + Ej=i,j^igijy{Pj) 

where y{-) represents the modeling of lower and upper bounding the transmit power and of 
any other effects e.g. power amplifier, etc. For example, y{pi) = max min {pmax, Pi}} or 
corresponding piecewise linear function y{pi) = \pi + Pmax\ — \Pi — Pmax\ yields eq.(2). 

By relaxing the positivity conditions in the power control problem in (3) and using sigmoid as 
the bounding function to the states in the denominator, and a different function in the nominator, 
the following fictitious "SIR" is defined in [12]: 



~ei = ^-^^p^^^^^^r^, z = l,...,7V (4) 



where 9i is the defined fictitious "SIR", Xi is the state of the i'th neuron, an is the feedback 
coefficient from its state to its input layer, Wij is the weight from the output of the j'th neuron to 
the input of the j'th neuron, and /2(-) represents the sigmoid function, and /3(-) represents the 

function used for self-state-feedback. Sigmoid function is defined as foiei) = 1 — t, r \, 

where cri > is called slope of /2(-), which is equal to its derivative with respect to its argument 
at the origin 0. 

It's shown in [12] that choosing /3(-) as a unity function in (4), i.e., f2{xi) — Xi, yields a 
network, called Sgm"SIR"NN which exhibits similar features as Hopfield NN does. So, following 
fictitious "SIR" is defined 
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(5) 



which is shown to satisfy the equilibrium points (prototype vectors) of the following dynamic 
network with 6^ = h called Sgm"SIR"NN in [12]: 



x = fi(- Ax + Wf2(x)+b) 
where x represents the derivative of x with respect to time and 



(6) 
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(7) 



In eq.(7), A shows the self- state-feedback matrix with ajj > 0, W with zero diagonal shows 
the connection weight matrix from outputs to other neuron's inputs, and b is a threshold vector. 

It's shown in [12] that the network in (6) exhibits similar features as continuous Hopfield 
Network does. In this paper, we examine its discrete-time version. 

From the fictitious CIR definition in eq.(5), let's define the following error signal 



N 

ii = -aiiXi + Ii, where li = k ^ Wijf2{xj), i = l,...,N 



(8) 



Writing (8) in matrix form gives 



e= -Ax + Wf2(x) + b (9) 

which is equal to the argument of the fi(-) in the network Sgm"SIR"NN in eq.(6). 

From eq. (6) and (9), x = e. If ej = given that Xi Q and /j 7^ 0, then, from eq.(5) and 

(8), = 9f = 1. 

The prototype vectors are defined as those x's which make 6i = 9^ = 1, i = 1, . . . , N 
given that Xi ^ and li ^ 0. So, from (4) and (5), the prototype vectors make the error signal 
zero, i.e., Cj = 0, i — 1, . . . , N. 
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A. Discrete Sgm" SIR" NN Network 

In this section, we present a Sigmoid based "SIR" -balancing network which exhibits similar 
features as discrete Hopfield NN does. 

Discretizing the differential equation (6) by the Euler method gives 

x^+i = - afi ( - Ax'^ + Wf (x^) + b) (10) 

where A, W and b are defined as in eq.(7), and k represents the iteration step. 
From eq.(lO) and (7), 

TV 

4+i = 4 + Q;Vi(-ai,4 + &,+ E ^ijUx^)) J = 1,...,N (11) 

where is the step size at time k. 

We will call the network in eq.(ll) as D-Sgm"SIR"NN (Discrete Sigmoid "SIR"-balancing 
neural network). 

The performance index is defined as Zi-norm of the error vector in (9) as follows 

N 

Vik) = \\e{k)\U = Elei(fc)l (12) 

i 

N N 

= J2\ ~ + where /j = 6j + J2 ^ijMxj) (13) 

In what follows, we examine the evolution of the the energy function in (12) in synchronous 
and asynchronous work modes. Synchronous mode means that at every iteration step, at most 
only one state is updated, whereas asynchronous mode refers to the fact that all the states are 
updated at every iteration step according to eq.(ll). 

Proposition 1: 

In asynchronous mode, in the D-Sgm"SIR"NN in eq.(ll) with a symmetric matrix W, the 
Zi-norm of the error vector in eq.(12) decreases at every step for a nonzero error vector, i.e., the 
error vector goes to zero for any such that 

|ej| > |a,-,aVi(eJ)| (14) 
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if 



N 



\W. 



(15) 



\ajj\ >k2 J2 

where k2 = 0.5a is the the global Lipschitz constant of /2(-) as shown the in Appendix A. 
Proof: 

In asynchronous mode, only one state is updated at an iteration time. Let j shows the state 
which is updated at time k whose error signal is different than zero, i.e., ej = —ajjXj + Ij ^ 
0, where Ij — bj Y.f=i^ijLj Wjif2{xi), as defined in eq.(8). 

Using eq.(9), we get 
















+ 














- /2(4)) 







Using the error signal definition of eq.(8) in eq.(ll) gives 



(16) 



So, the error signal for state j is obtained using eq.(16) and (17) as follows 



(17) 



k+l _pk _ 



-ajj{Xj~^ 



x]) 



From eq.(18) and (19), if a is chosen to satisfy |e^| > \ajjafi{e^)\, then 



,fc+i| 



< |e^|, for|ef I ^ 



(18) 
(19) 



(20) 



Since sigmoid function /i(-) is an increasing odd function and fi{ej) — if and only if = 0, 
then it's seen that there a can easily be chosen small enough to satisfy |e^| > aajj\fi{e^)\ 
according to the parameter ajj and slope of sigmoid function /i(-). 
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Above, we examined only the state j and its error signal ej{k). In what follows, we examine 
the evolution of the norm of the complete error vector e'^"'"^ in eq.(16). From the point of view of 
the h norm of the e*^"*"^ , the worst case is that while | | decreases, all other elements | | , i ^ j, 
increases. So, using eq.(16), (18) and (20), we obtain that: If 

N 

I -a,,(4+i- 4)1 > 1/2(4+1) E Kl (21) 

then 



|e(A; + l)||i< 



<||e(A;)||i if||e(A;)||i^O 
-0 if||e(fc)||i - 



(22) 



The sigmoid function /2(-) is a Lipschitz continuous function as shown in Appendix A. So, 

k2\x';+' - 41 > 1/2(4+1) - /2(4)| (23) 

where k2 = O.Sa is /2(-)'s global Lipschitz constant as shown in Appendix A. 
From eq.(21) and (23), choosing \ajj\ > k2\J2iLi,{ij^j)\wij\ yields eq.(21), which implies 
eq.(22). This completes the proof. 

■ 

Proposition 2: 

In asynchronous mode, choosing the slope of /2( ) relatively small as compared to /i( ) and 
choosing ajj > and a satisfying (14), the D-Sgm"SIR"NN in eq.(ll) with a symmetric matrix 
W is stable and there exists a finite step number such that the /i-norm of the error vector 
in eq.(12) goes to zero as its steady state. If 6i — 9^^*^ = 1 at the converged point, then it 
corresponds to a prototype vector as defined above. 
Proof: 

Since it's asynchronous mode, eqs.(16)-(20) holds where a^j > 0. So, if a!' at time k is chosen 
to satisfy \e^j \ > \ajja'^fi{ej)\ as in (14), then 

\ej{k + 1)1 < \ej{k)\, for \ei{k)\ ^ (24) 
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Note that it's straighforward to choose a sufficiently small a'^ to satisfy (14) according to ajj 
and the slope a of sigmoid /i(-). 

Using eq.(16), (18) and (24), it's seen for e^^ that: If 

|-a,,(4+^-4)| = \-a,,ah{e))\ (25) 

N 

> mxY') - f2{x])\ y: k^.i (26) 

then 

||e(A; + l)||i < ||e(A;)||i (27) 
We observe from eq.(18), (25), (26) and (27) that: 

1) If the xf, i = 1,...,N, approach to either of the saturation regimes of its sigmoid 
function /2( ), then 

1/2(4+') -/2(4) I E \wij\^0, j^l,...,N (28) 

since |/2(4^^) -/2(4) I ~ 0> i = 1, . . . , A^. That makes eq.(25) and (26) hold. Therefore, 
the norm of the error vector in eq.(12) does not go to infinity, and is finite for any x. 

2) x{k + 1) = x{k) if and only if e(A;) = 0, i.e., 

xj+i = if and only if /i(ej) = 0, j^l,...,N (29) 

3) Examining the eq.(17), (18) and (19) taking the observations 1 and 2 into account, we 
conclude that any of the x^, j = 1, ... ,7V, does not go to infinity, and is finite for any k. 
So, the D-Sgm"SIR"NN in eq.(ll) with a symmetric matrix W is stable for the assumptions 
in proposition 2. Because there is a finite number of in-saturation states, (i.e. the number of all 
possible in-saturation state combinations is finite), which is equal to 2^, there exists a finite step 
number, say Td, such that e{t) = for any t > T^. 

From eq.(5), if 9i = Oj^^^ = 1 at the converged point, then it corresponds to a prototype vector 
as defined in previous section, which completes the proof. 
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In what follows, we examine the evolution 6'^. From eq.(5), by choosing Of^ — 1 , let's define 
the following error signal at time k 

^^^-9^ + ef = -e^ + i, 3^1,. ..,N (30) 

Lemma 1: 

In asynchronous mode, in the D-Sgm"SIR"NN in eq.(ll) with a sufficiently small and 
with a symmetric matrix W, the is getting closer to 9*f*' — 1 at those iteration steps k where 
^ 0, i.e., |^j(A; + 1)| < \ij{k)\, where index j shows the state being updated at iteration k. 
Proof: 

Let j shows the state which is updated at time k. The fictitious "SIR" is defined by eq.(5) for 
nonzero as follows 

^J = ^' where /j= = + E w,,f{x^) (31) 
Let's define the following error signal in 9j, named ^j, as follows 

«l = -«) + l=^^4^ (32) 

j 

In asynchronous mode, from eq.(31), = ^m^^- Using this observation and eq.(32) 

j 

From (11) and (33), 

_ = Z^!!^^ (34) 



Provided that I'- ^ 0, we write from eq.(8) and (32) 



(35) 



Writing eq.(35) in (34) gives 
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-^J- ^^^""^f^^^^ (36) 
j 



From (36), since sigmoid function /i( ) is an odd function, and an > and a > 0, 



= - where /3 = | ^-^^'^^ | (37) 

j 

As seen from eq.(37), for a nonzero choosing a sufficiently small a satisfying > /? 
assures that 



lefVK'l if ^^v^O (38) 

which completes the proof. 

■ 

Proposition 3: 

In asynchronous mode, provided that the D-Sgm"SIR"NN in eq.(ll) with a symmetric matrix 
W converges to one of the prototype vectors according to proposition 1 and 2, 

e-=l, 3 = l,...,N (39) 

if and only if 0, j — 1, . . . , N for the converged prototype vector. 
Proof: 

Proposition 1 and 2 shows that the norm of decreases and approaches to zero in a finite step 
number and lemma 1 shows the norm of also decreases if /f ^ 0. From eq.(35), ef = Xf^f : 
As the ef approaches to zero, then also approaches to zero, given that ^ 0. This is sketched 
by the following equation 



ef = 0, {i.ej\ = \) if e^ = 0, and J, (40) 

(41) 

On the other hand, if = at the converged fixed point, then e*^ = because from eq.(35), 

,fe iktk 
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ef = if ef = 
provided that hj^O, j — 1, . . . , N, which completes the proof. 



(42) 



Proposition 4: 

The results in proposition 1 and 2 for asynchronous mode hold also for synchronous mode. 
Proof: 

In asynchronous mode, from eq.(9) 








N 



i=l 









Wii 




W2i 


+ 














Using (8) in eq.(43) and writing elementwise gives 



N 



6j — Cj CiiiOi 



From eq.(43) and (44), we obtain 



(43) 



(44) 



I - (xf +^ - ) I = I - auafi (ef ) | (45) 
> \f2{x^^') - f2{x1)\ E i-h...,N (46) 

which is equal to (21) in proposition 1 and (25) in proposition 2. Continueing the the steps of 
the analysis in proposition 1 and proposition 2 yield the results in proposition 1 and proposition 
2 respectively. 
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B. Fixed-Step Discrete" SIR" NN Network (FS"SIR"NN) 

In this subsection, establishing an analogy to the traditional fixed step 1-bit increase/decrease 
power control algorithm e.g. [22], [23], we propose the following network by replacing the /i(-) 
in eq.(ll) by sign function as shown in the following 

N 

x']+^ = x^j+Asign(^-ajjX^j+bj+ '^M^j)) J = ^,---,N (47) 

where /2( ) represents the sigmoid function. We call the network in eq.(47) as Fixed-Step 
"SIR" Neural Networks (FS"SIR"NN). 

Corollary 1: 

In the FS"SIR"NN in eq.(47) with a symmetric matrix W, the /i-norm of the error vector in 
eq.(12) converges to the interval [—aaa^+ana] while the xf converges to the interval [q;,+Q!] 
within a finite step number in asynchoronous mode if 

N 

\ajj\ >k2 Yl (48) 

where k2 — 0.5a is the the global Lipschitz constant of /2( ) as shown the in Appendix A. 
Proof: 

We're going to obtain the results by writing /i(-) = sign{-) in the proof of proposition 1 in 
section II- A above. This would correspond to a sigmoid function /i(-) whose slope is infinity 
in proposition 1. 

Let j show the state which is updated at time k. Following the steps in eq.(9) and (16) and 
writing /i(-) = sign{-) in eq.(17) gives 

-x^j = asign{e^) (49) 
So, the error signal for state j is obtained using eq.(16) and 49 as follows 



k+i _ k 

^3 ^3 



—ajjasign{e^) 



(50) 
(51) 
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From eq.(50) and (51), 

( > (52) 

I < \aajj\ otherwise 

Above, we examined only the state j and its error signal ej{k). In what follows, we examine 
the evolution of the norm of the complete error vector e'^"'"^ in eq.(16). From the point of view of 
the h norm of the 6^^+^, the worst case is that while |e^| decreases, all other elements |ef |, i j, 
increases. So, using eq.(16), eq.(50-(52), we obtain that: If 



N 

-a,,(4+i-xj)|> 1/2(4+1) -/2(4)| E \w,j\ (53) 



then 



|e(. + 1)11, r 'f|l-WII.>^E£M....I««l (54) 

< ^Yli^=i,{i^j)\aii\ otherwise 

The sigmoid function /2( ) is a Lipschitz continuous function as shown in Appendix A, 

hlxj'^'^ - x'^l > |/2(x^+i) - f2ix'-)\, where k2 = 0.5a is /2(-). From eq.(53) and the Lipschitz 
inequality, choosing \ajj\ > k2\J2i^=i,(ijLj) satisfies eq.(53), which implies eq.(54). This 
completes the proof. 



Corollary 2: 

In asynchronous mode, choosing /i(ej) = sign{ei), ajj > and a > 0, the FS"SIR"NN in 
eq.(47) with a symmetric matrix W is stable and there exists a finite step number such that 
the ii-norm of the error vector in eq.(12) converges to the interval [—ana, +aiia] while the 
converges to the interval [a, +a\ within a finite step number. 
Proof: 

Writing /i( ) = sign{-) in proposition 2 in section 11- A and following the steps and the 
observations therein gives that the FS"SIR"NN in eq.(47) is stable and there exists a finite 
step number such that the /i-norm of the error vector in eq.(12) converges to the interval 
[—aiia,+aiia] while the x^ converges to the interval [q:,+ci;]. 
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Corollary 3: 

The results in corollary 1 and 2 for asynchronous mode hold also for synchronous mode. 
Proof: 

Writing ) = sign{-) in proposition 1 and 2 in section II-A and following the the steps and 
observations of the analysis as in proposition 4 in section II-A for synchronous mode yields the 
results in corollary 1 and 2 respectively above. 

■ 

It's known form literature that the performance of Hopfield network may highly depend on 
the parameter setting of the weight matrix (eg. [15]). There are various ways for determining 
the weight matrix of the Hopfield Networks: Gradient-descent supervised learning (e.g. [9]), 
solving linear inequalities (e.g. [20], [21] among others), Hebb learning rule [19], [14] etc. How 
to design CINR-SgmNN is out of the scope of this paper. The methods used for traditional 
Hopfied NN can also be used for the proposed networks D-Sgm"CIR"NN and FS"CIR"NN. 



We take the same examples as in [12] for comparison reasons and for the sake of brevity 
and easy reproduction of the simulation results. In [12], the performances of continuous-time 
networks, Sgm"SIR"NN and Hopfield networks, are examined. In this paper, their discrete-time 
versions are examined. We apply the same Hebb-based (outer-products-based) design procedure 
([19]) in [12], which is presented in Appendix B in case of orthogonal prototype vectors. 

In this section, we present two examples, one with 8 neurons and one with 16 neurons. The 
weight matrices are designed by the outer products-based design in Appendix B. 

As in [12], traditional Hopfield network is used a reference network. The discrete Hopfield 
Network [1] is 



where W is the weight matrix and is the state at time k, and at most one state is updated. 
Example 1: 

In this example, there are 8 neurons. The desired prototype vectors are 



III. Simulation Results 




(55) 
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D 



11 1 1-1-1-1-1 

11-1-11 1-1-1 
1-11-11-11-1 



(56) 



The weight matrices A and W, and the threshold vector b are obtained as follows by using 
the outer-products-based design presented in Appendix B and the slopes of sigmoid functions 
/i(-) and /2(-) are set to cti = 10 and (72 = 2 respectively, and p = 0, a = 0.1 and A = 0.1. 



A = 31, W = 
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(57) 



1 -1 -1 -3 
-1 1 -3 -1 
-1 -3 1 -1 
-3 -1 -1 1 

The Figure 1 shows the percentages of correctly recovered desired patterns for all possible ini- 
tial conditions x'^ e (-1, +1)^, in the proposed networks D-Sgm"SIR"NN and FS-Sgm"SIR"NN 
as compared to traditional discrete Hopfield network. 

Let nid show the number of prototype vectors and C{N, K), (such that > -f^ > 0), represent 
the combination N, K, which is equal to C(7V, K) = jjf^^^, where ! shows factorial. In our 
simulation, the prototype vectors are from (—1,1)^ as seen above. For initial conditions, we 
alter the sign of K states where K=0, 1, 2, 3 and 4, which means the initial condition is within 
K-Hamming distance from the corresponding prototype vector. So, the total number of different 
possible combinations for the initial conditions for this example is 24, 84 and 168 for 1, 2 and 
3 -Hamming distance cases respectively, which could be calculated by rud x C{8,K), where 
rrid — S and K — I, 2 and 3. 

As seen from Figure 1 the performance of the proposed network D-Sgm"SIR"NN is remark- 
ably better than that of traditinal discrete Hopfield NN for 1, 2 and 3 Hamming distance cases. 
The FS"SIR"NN also considerably outperforms the Hopfield for 1 and 2 Hamming distance 
cases while Hopfield NN outperforms FS"SIR"NN at 3 Hamming distance case. 
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Example 2: 

The desired prototype vectors are 



D 



(58) 



11 1 1 1 1 1 1-1-1-1-1-1-1-1-1 
11 1 1-1-1-1-11 1 1 1-1-1-1-1 
11-1-11 1-1-11 1-1-11 1-1-1 
1-11-11-11-11-11-11-11-1 
The weight matrices A and W and threshold vector b is obtained as follows by using the 
outer products based design explained above. 
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(59) 



The Figure 2 shows percentage of correctly recovered desired patterns for all possible initial 
conditions x'^ e (-1, +1)^6, in the proposed D-Sgm"SIR"NN and FS"SIR"NN as compared to 
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discrete Hopfield network. 

The total number of different possible combinations for the initial conditions for this example 
is 64, 480 and 2240 and 7280 for 1, 2, 3 and 4-Hamming distance cases respectively, which 
could be calculated by x C(16, K), where rrid — A and K — I, 2, 3 and 4. 

As seen from Figure 2 the performance of the proposed networks D-Sgm"SIR"NN and 
FS"SIR"NN is the same as that of discrete Hopfield Network for 1 -Hamming and 2-Hamming 
distance cases (%100 for all networks). However, the D-Sgm"SIR"NN and FS"SIR"NN gives 
better performance than the discrete Hopfield network does for 3 and 4 Hamming distance cases. 

IV. Concluding Remarks 

This paper is continuation of the work in [12] where we present and analyse a Sigmoid- 
based "Signal-to-Interference Ratio, (SIR)" balancing dynamic network in continuous time. In 
this second part, we present the corresponding network in discrete time: We show that in the 
proposed discrete-time network, called D-Sgm"SIR"NN, the defined error vector approaches to 
zero in a finite step in both synchronous and asynchronous work modes. Our investigations show 
that i) Establishing an analogy to the distributed (sigmoid) power control algorithm in [10] and 
[11], if the defined fictitious "SIR" is equal to 1 at the converged eqiulibrium point, then it is 
one of the prototype vectors, ii) The D-Sgm"SIR"NN exhibits similar features as discrete-time 
Hopfield NN does, iii) Establishing an analogy to the traditional 1-bit fixed-step power control 
algorithm, the corresponding "1-bit" network, called Sign"SIR"NN network, is also presented. 
Computer simulations show the effectiveness of the proposed networks as compared to traditional 
discrete Hopfield Network. 

Appendix A 

In what follows, we will show the sigmoid function (/2(a) = 1 — i^exp{-aa) ' cr > 0) has the 
global Lipschitz constant k = O.ba. 

Since /(•) is a differentiable function, we can apply the mean value theorem 

/(a) - f{b) = {a- b)f'{i,a + (1 - - a)) 

with fj, e [0, 1] 
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The derivative of /(■) is /'(a) = ^aa^i'^gaay2 whose maximum is at the point a = 0, i.e., 
I /'(a) I < 0.5a. So we obtain the following inequality 

\f{a)-m\<k\a-b\ (60) 
where A; = O.Scr is the global Lipschitz constant of the sigmoid function. 

Appendix B 
Outer products based network design: 

Let's assume that L desired orthogonal prototype vectors, {d,}fLp are chosen form (—1, +1)^. 
Step 1: Calculate the sum of outer products of the prototype vectors (Hebb Rule, [19]) 

Q=x:d,dj (61) 

s=l 

Step 2: Determine the diagonal matrix A and W as follows: 

i,j^l,...,N (62) 



aij = < 



qu + p if i^j, 
if iy^j 

where p is a real number and 



i,j = l,...,N (63) 



if i^j, 
^ Qij if J 

where qij shows the entries of matrix Q, is the dimension of the vector x and L is the 
number of the prototype vectors (N > L > 0). In eq.(62), qa = L from (61) since {ds} is 
from (—1, +1)^ and p is a real number. However, from the analysis in section II-A and II-B, it 
can be seen that the proposed networks D-Sgm"SIR"NN and FS"SIR"NN contain the prototype 
vectors as their equilibrium points for a relatively large interval of p. 

Another choice of p in (62) is p — N — 2L which yields an — N — L. In what follows we 
show that this choice also assures that {dj}^^^ are the equilibrium points of the networks. 

From (61)-(63) 

L 

[-A + W] = -(iV-L)I + ^d,df -LI (64) 
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where I represents the identity matrix. 

Since e (—1,+!)^, then \\ds\\l — N. Using (64) and the orthogonality properties of the 
set {dJfLi gives 

[-A + W]d, = -{N - L)ds + {N- L)ds = (65) 
So, the prototype vectors {dj }^^i correspond to equilibrium points. 
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